Fox News Flash top headlines are here. Check out what's clicking on FoxNews.com. It's called a biometric gate check -- more commonly known as facial recognition technology. U.S. Customs and Border Protection (CBP) has used it to process more than 100 million travelers at airports in the U.S. But now a bipartisan pair of U.S. senators is asking how the data is being used and trying to determine if it's an invasion of privacy, similar to daily life in communist China.

Masked language modeling (MLM) pre-training models such as BERT corrupt the input by replacing some tokens with [MASK] and then train a model to reconstruct the original tokens. This is an effective technique which has led to good results on all NLP benchmarks. We propose to expand upon this idea by masking the positions of some tokens along with the masked input token ids. We follow the same standard approach as BERT masking a percentage of the tokens positions and then predicting their original values using an additional fully connected classifier stage. This approach has shown good performance gains (.3\% improvement) for the SQUAD additional improvement in convergence times. For the Graphcore IPU the convergence of BERT Base with position masking requires only 50\% of the tokens from the original BERT paper.